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ABSTRACT 

The Portland (Oregon) Public Schools Research and 
Evaluation Department is investigating the use of evaluative data in 
instructional planning and decision making in schools. The three-year 
research effort involves training and technical support in effective 
evaluation use. The first year of the evaluation use research focused 
on collecting and analyzing baseline data on the use of Portland 
Achievement Levels Tests (PALT). Interviews and surveys gathered 
information from parents, teachers, principals, and directors of 
instrumentation on how these actors use evaluative data in 
instructional planning and decision-making. Recommendations from the 
first-year study include: (1) revision of parent conference report 
forms to include numeric and graphic test score summer ier; (2) 
revision of the parent guide to understanding test scores; (3) 
institution of a spring parent conference report to include fall and 
spring PALT scores; (4) evaluation of the grade 3 testing program, 
revised on the basis of teacher interviews and surveys; (5) 
development of training materials to help teachers prepare students; 
and (6) development of inservice training for test coordinators and 
others interested in test interpretation and the use of results. 
Summary results of surveys of parents and teachers are appended. 
(TJH) 
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AnONISTRATIVE SUMMARY 



THE USE OF EVALUATIVE DATA FOR 
INSTRUCTIONAL PLANNING AND DECISION MAKING, A RESEARCH STUDY 

1986-87 

The Portland Public Schools Research and Evaluation Department is 
investigating the use of evaluative data in instructional 
planning and decision making in schools. The three year research 
effort developed from the department's commitment to improve 
evaluative data use at all levels of the decision making process. 

The assumption of the research is that the purpose of evaluation 
is to inform decision making. Eflective evaluation use should 
result In better instructional decisions for students. Improved 
decision making should result in increased achievement, more 
efficient use of school resources and greater satisfaction on the 
part of students, parents, teachers and principals. 

This research and development effort will assist the district to 
enhance the use of evaluation. Training and technical support in 
effective evaluation use will be developed based cn stated needs 
of participants and findings from the research. 

The first year of the evaluation use research focused on 
collecting and analyzing baseline data on the use of Portland 
Achievement Levels Tests (PALT) in the district. Interviews and 
surveys gathered information from parents, teachers, principals, 
and Di\*ectors of Instruction on how these actors use evaluative 
data in instructional planning and decision making. 

Recommendations from the first year of the study include: 

1. The R&E Department should revise the Parent Conference 
Report. Consideration should be given to including numeric 
and graph c test score summaries. A revised Parent Guide to 
understanaing test scores should accompany the report. 

2. The R&E Department should institute a Spring Parent 
Conference Report , including fall and spring PALT test 
scores. The report could be mailed to parents with the 
Parent Guide to testing. 

3. Based on early findings from teacher interviews and surveys, 
the R&E Department revised the grade 3 testing program. 
Program implementation and changes should be evaluated. 

4. The R&E Department should consider developing training 
materials/packages to help teachers prepare students for 
testing. Interviews identified this area as a priority 
interest of teachers ani principals. 

5. The R&E Department should consider developing inservice 
training for test coordinators or others interested in test 
interpretation and use of test results. 
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THE USE OF EVALUATIVE DATA FOR INSTROCTIONAL PLANNING 
AND DECISION MAKING IN SCHOOLS: YEAR ONE REPORT 
July 1986 - August 1987 

INTRODOCnON 

The major purpose of achievement testing in schools is to inform 
educational decisions. Toward this end, testing and the subsequent 
reporting of student outcomes, have been a long-standing tradition 
in schools. While schools recognize the need for a close fit 
between curricultoi and testing r in practice, they often don't use 
test results to improve instructional decisions. Thus, evaluation 
use may vary widely with the interest and experience of the user. 

In 1986, the Research and Evaluation Department in Portland Public 
Schools began a district-wide study to exeuaine evaluation use in 
schools. This research has the following objectives: 

1) to provide information on the extent to which evaluation data 
are used by principals and teachers in instructional planning 
and decision making, and 

2) to develop improved reporting and support systems, 
including the development of training materials to 
improve evaluation use at classroom and building level « 

The research assumes that the goal of evaluation is to inform 
decision making. The effective use of evaluation should result in 
better decisions aUbout what to teach, who to teach, and how to 
teach. Improved decisions should result in increased achievement, 
more efficient use of school resources and greater satisfaction on 
the part of students, parents, teachers and principals. 

This report siommarizes research for the first year of the decision 
making study. The report is intended for use by the Director of 
Research and Evaluation and Evaluation Department staff as an aid 
in improving the structure and operation of the testing program and 
developing improved technical support for the Portland Public 
Schools. It is expected to also be of interest to district 
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administrators, school board members, and researchers with an 
interest in evaluation use and its impact on educational practice. 

RESEARCH PLAN/QUESTIONS 

This research will collect information on the following questions: 

1. What is the decision making process for instructional 
planning in schools? 

2. Hov do schools use evaluative report data in instructional 
planning and decision making? 

3. What is the level of use of evaluation data in instructional 
planning and decision making in the Portland Public Schools? 

4 • What is the relationship between student achievement and 
evaluation use? 

5. How can measurement and evaluation most effectively serve 
the decision making needs of parents, teachers, principals. 
Directors of Instruction, and central office staff? 

The research and development process began in the summer of 1986 
and will continue over a three year period. It should result in 
improved reporting and delivery systems for Portland Achievement 
Levels Test (PALT) data, including training and support in 
evaluation use to improve instructional decision making. The study 
consists of four phases of activity: 1) conducting the evaluation 
use study, 2) developing training materials, 3) training in 
evaluation use, and 4) follow-up. 

Evaluation ase study 

The evaluation use study consists of three parts - surveys on 
evaluation use, focused interviews on the use of evaluation 
reports, and case studies of the use of evaluation in schools. 

In fall 1986, elementary school principals and teachers 
participated in surveys on evaluation data use in instructional 
planning. Parents were surveyed on their use of achievement data 
in the Parent-Teacher Conference Report. Focused interviews were 
conducted with 12C teachers, principals, and administrators. The 
data have already begun to be used in Departmental planning. 

- 2 - 
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In fall 1987, we'll conduct in-depth case studies to document the 
types of decisions made based on evaluation data at classroom and 
building level. This activity will develop a description of the 
decision process and informational needs at different levels in the 
system. The knowledge gained through this process will help the 
Research and Evaluation staff develop a richer understanding of 
uses of and needs for evaluative data including, but not limited 
to, Portland Achievement Levels Tests (PALI) data. 

Development of Trainina/Supnorli Mai-^T-ials 

Research findings will guide the development of a series of 
training modules and other support materials for principals and 
teachers on effective evaluation use. First modules will be aimed 
at awareness; later modules will be targeted to meet stated needs 
of teachers and administrators. Material formats may include 
newsletters, packaged materials, and inservice presentations. 

Training 

Training or training packages will be provided to a select group of 
principals and teachers who participated in the evaluation use 
study. Participants will be randomly reassigned to two equal sized 
groups; one group will receive training and one group will not. 
This phase of the study will continue through 3 989 as new materials 
are developed. 

Follow-up. evaluation and m onitoring 

As training is conducted, data will be collected from the treatment 
and no-treatment groups on: 1) value/effectiveness of the training, 
2) changes in job satisfaction, and 3) student achievement. This 
Information will be used to improve the training materials, 
delivery system and reporting process. 



- 3 - 



ERIC 



8 



BACKGROUND LITERATORE 



Evaluation use for instructional decision making is a highly 
complex process. Two key factors which affect evaluation use in 
schools are the individuals^ proficiency in evaluation and their 
attitude or predisposition toward using evaluation information 
(Pechman and King, 1984). Principals and teachers bring a variety 
of prior kpowledge t their instructional decisions (Alkin, 
Daillak, and White, 1980). Many have limited experience with 
evaluation and rely instead on informal assessments or teacher 
judgement for decision making. Teachers and principals with 
limited experience in using data often need to gain awareness of 
evaluation and its role in instructional decision making. 

Understanding how teachers respond to change efforts in schools has 
been the research focus of the Concerns-Based Adoption Model (CBAM) 
(Hall, Wallace, and Dossett, 1973). The model suggests that as 
individuals are introduced to an innovation, they pass through 
predictaUble stages in attitude and behavior. The stages indicate 
developmental movement from awareness, personal, management, 
consequence, collaboration, to refocusing in new directions. 

Though the Concerns-Based Adoption Model is well accepted in change 
process literature, it has received limited use in research on 
evaluation use. Pechman and King (1986) used the CBAM levels of 
use freunework to define the structure of school evaluation use 
(SSEU) . Hall (1982) proposed a preliminary list of key components 
for evaluation use as an innovatio«^. The purpose of sub-study 1, 
reported below, was to test the levels of use hypothesized in the 
CBAM, specifically the level of use of evaluation reports in 
Portland Public Schools. Sub-study 2 expanded on this with 
descriptive information on the use of evaluation and validation of 
the findings of the first study. Sub-studies 3 and 4 evolved from 
the earlier studies in an effort to better understand and explain 
the interview and survey results. 

- 4 - 
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methodolcx:y 



The 1986-87 evaluation use research utilized four sub-studies: 

1) Levels of Use interviews on the use of evaluation test reports, 

2] Surveys of principals and teachers on evaluation use, 

3) Survey of parents on the Parent-Conference Report, and 

4) Telephone survey of grade 3 teachers on the testing program. 

A random seunple of teachers and principals in 80 elementary schools 
participated in studies 1 and 2. The xrethodology and findings of 
the 1986-87 evaluation use studies are presented here. 

Sub-Studv 1 - Le vels of Use 

Subjects 

Levels of Use (LoU) interviews were conducted with 126 district 
staff across all clusters by nine certified LoU interviewers. 
Eight Directorb of Instruction, 82 teachers (grades 3-8), and 36 
principals participated in interviews focused on use of PALT 
evaluation reports. Subjects were randomly selected from 45 
elementary schools in the district. Subjects varied in years of 
experience in their position (one to twenty- five years) and the 
amount of evaluation experience (nonuser to evaluation specialise, . 

Measure 

In order to determine a baseline measure of use of evaluation 
reports in the district, the researchers used the Levels of Use of 
an Innovation methodology. The Levels of Use (LoU) model was 
developed by Gene Hall and Susan Loucks at the Research and 
Development Center for Teacher Education, University of Texas- 
Austin. The model identifies eight levels of use: nonuse, 
orientation, preparation, mechanical use, routine, refinement, 
integration , and renewal (Appendix A) . Zigure l outlines the 
levels of use and related behavioral characteristics as defined in 
the evaluation use study. The LoU model has been used successfully 
to study evaluation use ir other school districts (Pechman and 
King, 1986) . 

- 5 - 
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FIGURE 1. 

LEVELS OF USE AND BEHAVIORAL DEFINITIONS 



Level of Use Behavioral Definition of Evaluation Use 



0 NONUSE User has little or no knowledge of evaluation 

repoirts and no involvement with reports. 

1 ORIENTATION User has recently acquired information about 

evaluation reports or is exploring its value 
and its demands upon the user and user system. 

II PREPARATION User is preparing for first use of evaluation 

repozrts. 

III MECHANICAL USE User focuses on short term use of evaluation 

reports with little refection. Changes in use 
are made to meet user needs. User is engaged 
in a step-by-step attempt to master the task 
of using evaluation reports, often resulting 
in disjointed and superficial use. 

Stable use of evaluation reports. Few changes 
are made in use. Little thought is given to 
improving evaluation use or its consequences. 

User varies the use of evaluation reports to 
increase impact on students. Variations are 
based on knowledge. of both short and long-term 
consequences for students. 

User combines own efforts to use evaluation 
reports with activities of colleagues to 
achieve a collective impact on students. 

User reevaluates the quality of evaluation 
reports ; seeks major modifications or 
alternatives to current reports to increase 
student impact , examines new developments in 
the field, and goal for self or the system. 



IVA ROUTINE 



IVB REFINEMENT 



V INTEGRATION 



VI RENEWAL 



Procedure 

To measure Levels of Use, focused interviews were conducted. 
Interviewees were asked a series of branching inter'/iew questions 
and follow-up probes. Interviews took approximately twenty minutes 
and appeared to be informal conversations about how the principal 
or teacher is using evaluation reports. The LoU interview protocol 

- 6 - 
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is generic; it can describe different innovations by changing the 
point of reference. For our purpose, the innovation was use of 
Portland Achievement Levels Test (PALT) evaluation reports, 
specifically student achievement or administrative reports. 

The LoU interviews were tape recorded and the resultant tapes were 
rated for reliability by trained interviewer/ raters. Two raters 
listened to each tape and independently categorized rhe interviewee 
according to Level of Use of evaluation. If the two raters did not 
agree on the LoU, the tape was listened to by a third rater. 
Overall LoU was determined when two raters agreed on the rating. 

Of the total 126 LoU tapes, 106 or 84% were agreed on by the first 
two raters. Of the remairing 20 tapes, 16 or 13% were classified 
by the third rater. Only 4 or 3% required a fourth rater for a 
final rating. Inter-rater reliability ranged from .82 to .98 on 
the overall Level of Use. 

This research views the individual teacher or principal as the 
primary unit of evaluation use. The secondary unit of evaluation 
use is the school. The individual user is also the primary unit of 
analysis. Experience demonstrates that asking more removed sources 
(e.g., D.O.I.s, superintendents) aoout the use or nonuse of 
innovation by their staff is questionable. The only way to know 
if, and how, an innovation is being used is to directly assess each 
individual's use. The Levels of Use method allows us to focus on 
what individuals are doing in relation to using evaluative data. 

Results 

Figure 2 summarizes the Levels of Use of PALT reports. Figure 3 
presents the Levels of Evaluation Use of Portland teachers and 
administrators during 1986-87. The LoU process found that overall 
88% of the Directoi-s of Instruction (DOIs) , 97% of principals, and 
85% of teachers were users of PALT evaluation reports. We found 
seven percent of the sample to be nonusers of PALT evaluation data. 

~ 7 - 
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Figure 2. Levels of Use 

Um of PALT Evaluation Reports (n«126) 
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Rgure 3. Level of Us*, by School Role 

Use of PALT Evaluation Reports 
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LoU VI- Renewal 



Us 



Among individual users, 32% are Mechanical Users of PALT evaluation 
reports. This group needs additional training/support in the use 
and interpretation of test reports. The largest group, 40% of the 
sample, were identified as Routine Users of evaluation reports. 
This is solid, stable comfortable use of PALT information. Routine 
use is what we want people to achieve as knowledgeable users. The 
interviews also identified a group of '^enhanced users", individuals 
at Refinement (Lou IVB) , Integration (LoU V) or Renewal (LoU VI) . 
The ''enhanced users**, 21% of the sample, are comfortable with using 
PALT reports and often make refinements to increase student impact. 
Principals were more likely to be higher level user^: of PALT 
reports than teachers. While 36% of the principals "enhanced 
users" of the reports, only 13% of teachers are among this group. 

Analysis of LoU in 34 school buildings which had at least three 
individual LoU ratings. Indicated that 10 schools or 29% are using 
PALT reports at a Mechanical Use level, 11 or 32% are using 
evaluative data at Routine level, and only 2 or 6% ^re "enhanced" 
users of PALT evaluation reports. 

The authors hypothesized a positive relationship between level of 
use of evaluative data and increased student achievement. We 
expected different student outcomes at different levels of use. 
For example. Mechanical Users (LoU III) are struggling with 
logistics and management problems associated with new use of 
evaluation reports Higher level users, those at Refinement (LoU 
IVB) , are adapting their use of evaluation reports to increase 
student impact. Mechanical users would have less effect on student 
achievement than Routine users and Refinement users would likely 
have more positive results than either Mechanical or Routine users. 
If this hypothesis held, there would be an increasing relationship 
between level of evaluation use and student achievement. 

h variety of statistical procedures were used to explore the 
relationship between level of use of PALT data within a building 

- 9 - 
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and student achievement. Although some relationships were 
observable within schools and grade groupings, no significant 
correlations were obtained. The exploratory analysis of student 
achievement suggests that the relationship is not predicated on the 
defined level of use, but may lie more directly in how the data are 
actually used in decision making. Further evidence for this will 
be sought in the case studies. 

Sub-atudv 2 - Principal and Teacher Surveys 

Subjects 

In November 1986, survey questionnaires were sent to a random 
sample of principals and teachers in 40 elementary schools. 
Surveys were completed by 35 principals and 239 teachers on their 
use of evaluative data in instructional decision making. 

Measure 

A Principal Survey and Teacher Survey (Appendix B) were developed 
to determine what kind of evaluation data are used by teachers and 
principals. The major questions the surveys asked were: 1) which 
PALT evaluation reports do you use to assist you in instructional 
decision making situations and 2) what test scores do you use in 
decision making situations. The surveys also asked about the use 
and usefulness of the current PALT reporting system, how test 
reports are used, and what other sources of evaluative information 
are used in instructional decision making. 
Procedure 

The surveys were mailed to a fifty percent random sample of 
.principals and teachers in November 1986. A cover letter 
describing the study and a preaddressed return envelope were 
included in the mailing. The survey response rate was 88% for 
principals and 84% for teachers after one follow-up wailing. 

Results 

Survey results were analyzed to determine what PALT evaluation 
reports are used, the extent to which RIT and P-scores are used, 

- 10 - 
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and how test scores are used in decision making situations by 
principals and teachers. Frequency tables for the major survey 
questions are r resented in Appendix B. 

What PALT evaluation re ports do principals and teachers use for 
Instructional planning and decision making? 

Table 1 displays frequencies of use of the PALT student achievement 
reports by principals and teachers. The results indicate that 
principals and teachers use evaluative data differently. More than 
half of the sample say they use five of the six achievement reports 
••almost always*' or "often**. Principals use the evaluation reports 
more frequently than teachers. Principals indicate they use the 
Fall-Spring Match report, Grade Goal Progress report, Student-Class 
Alpha report, and Labels for Cum Folder most frequently. Sixty-six 
percent of teachers use the Parent-Teacher Conference report almost 
always. Teachers and Principals use Test Score Cards least often. 
Sixty percent of the teachers seunpled report almost always or often 
using the student Goal Report -Class Alpha. 



TABLE 1. 

Fraquency of Use of Achievement Reports 



1 PALT Achievemenc 
1 Reports 


(School 
IRole 


N 


Almosc 
Always 


Often 


Some- 
cimes 


Seldom Almosc | 
Never | 


j Labels for 


1 Principal 


30 


70 


0 


3 


3 


20. 


0 


6 


7 




1 Cum. Folder 


1 Teacher 


196 


40 


8 


11 


7 


lb. 


8 


8 


7 


23.0 1 


1 Parenc-Teacher 


1 Principal 


32 


62 


5 


3 


1 


15. 


6 


12 


5 


6.3 1 


1 Confere le Rpc. 


[Teachet 


225 


66 


2 


16 


4 


8. 


9 


2 


2 


6.2 1 


1 Fall Spring 


1 Principal 


32 


59 


4 


25 


0 


12. 


5 






3.1 1 


1 Macch Reporc 


1 Teacher 


200 


42 


5 


26 


5 


18. 


0 


S 


5 


9.5 1 


1 SCudenc Goal 


[Principal 


34 


44 


1 


32 


4 


20. 


6 






2.9 1 


1 Class Alpha 


1 Teacher 


218 


32 


6 


27 


5 


24. 


8 


6 


9 


8.3 1 


1 Grade Goal 


(Principal 


32 


53 


1 


34 


4 


6 


3 


3 


1 


3.1 1 


1 Progress 


1 Teacher 


52 


26 


4 


23 


9 


27 


9 


8 


1 


13.7 1 


1 Tesc Score 


(Principal 


28 


17 


9 


14 


2 


32. 


1 


17 


9 


17.9 i 


1 Cards 


ITeccher 


182 


20 


9 


19 


2 


20 


9 


14 


8 


24.2 1 
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In the use of PALT administrative reports, 53% of principals use 
the Grade Goal Progress Report almost always. Principals least 
often use Board Report on Goal 2. Principals (33%) seldom or never 
use Achievement Report by Ethnic Group, Achievement Profiles, or 
Board Reports 1 & 2. 

What test scores fcvnes fRIT or P-scorel do teachers and principals 
usft in dq filfflon Bykina situations? 

Teachers and principals were asked what test scores they use in 
several decision making situations, including goal setting, student 
placement, curriculum evaluation, and parent/ teacher communication. 

In setting instructional goals , teachers use P-scores^ , but 
principals prefer RIT^ scores. Surprisingly, and of concern, we 
learned that 20% of principals and teachers don't think test scores 
are applicable to the instructional goal setting process. 

Regarding student placement/regrouping for instruction, both 
principals and teachers use RIT and P-scores in student placement. 
Seventy*four percent of the principals report using RIT & P-score 
for placement. Twenty-six percent of teachers rely on P-scores to 
help with placing students compared to 11% who use RIT scores. In 
evaluating curriculum, 28% of principals use RIT scores, while only 
10% of teachers use RITs in this decision making situation. 

The types of test scores used in communicating with teachers and 
parents is a highly charged area. Principals find P-scores more 
appropriate in talking with parents and staff. Forty- five percent 
of principals use RIT and P-scores in communicating with teacher. 

1 p-score is a standardized score with a mean of 50 and a 
standard deviation of 10. The P-score shows the student's 
standing compared to other Portland students in the same grade. 

2 RIT is the fiasch unit score. The RIT score shows growth on a 
grade to grade scale (ranging from 140 to 260) . The RIT score 
increases from grade to grade. 

- 12 - 
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Teacher survey results indicate three areas needing clarification 
for some teachers: 1) 18% of teachers feel test scores are not 
appropriate or meaningful when communicating with other teachers, 
2) 17% of teachers use P-score inappropriately for diagnosis and 
prescription of instruction, and 3) 20% of teachers feel that test 
scores are not applicable to instructional goal setting decisions. 

Analysis of the principal and teacher surveys by school type, 
elementary vs middle schools, fo\ind no differences in their use of 
RIT scores and P-scores. 

Table 2 presents the distribution of use of PALT and Non-PALT test 
score information by principals and teachers in Portland Public 
Schools. Overall, a higher percentage of principals xise PALT 
information than teachers. Both groups rely on PALT evaluation 
information than non*PALT scores. Overall, principals use RIT 
scores, while teachers feel more comfortable in using P-scores. 
Appendix B presents complete frequency tables from the surveys. 

TABLE 2, 

Frequency of Use of PALT and Non-PALT Test Scores 
By Principals and Teachers 



DECISION MAKING SCHOOL Non- TOTAL 

SITUATION ROLE PALT PALT N/A N % 



Goal seccing 


Principals 


82 


.8 






17 


.2 


29 


100 


.0 


by class/grade 


Teachers 


56 


.9 


26 


.1 


17 


.1 


211 


100 


.1 


Inicial scudenc 


Principals 


97 


.1 






2 


.9 


34 


100 


.0 


placemenc 


Teachers 


77 


.6 


13 


.2 


9 


.2 


228 


100 


.0 


Reviewing indlv. 


Principals 


97 


1 


2 


.9 






34 


100 


0 


student growth 


Teachers 


83 


.0 


12 


.6 


4 


.5 


223 


100 


1 


Regrouping 


Principals 


87 


9 


6 


.1 


6 


.1 


34 


100 


1 


students 


Teachers 


67 


0 


23 


4 


9 


.6 


218 


100 


0 


Curriculum 


Principals 
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Sub-atudv 3 - Parent: Survey 

In November 1986 ^ 27 3 parents of students in grade 3-8 completed a 
survey on their use of PALT student achievement test results. 
Because surveys were distributed during the Parent-Teacher 
Conference, the sample was limited to parents who attended the 
conference and voluntarily completed the ten item questionnaire. 

Measure 

The Parent Survey (Appendix C) was developed to determine how 
parents use achievement test data on the Parent Conference Report. 
Parents were also asked what evaluative information assists them in 
educational planning. The focus of the instrument was to determine 
the use and usefulness of the current Parent Conference Repoxrt and 
what revisions would make the report more valuab'ie to parents. 

Procedure 

The Parent Survey was distributed to parents at the Parent-Teacher 
Conference. The classroom teacher asked parents to complete the 
questionnaire and return it. Teachers returned the parent surveys 
to the researcher in the district mail service (Pony) . Questions 
asked parents for their reactions to the current Parent Conference 
Report. This survey provided the Evaluation Department with 
information for revising the parent conference report. 

Results 

Parent survey results (Appendix C) indicate that 66 parents or 25% 
of those responding find the current achievement test score 
information adequate. Of the parents sampled, 55 or 21% want more 
information on test scores and achievement ; another 2 1% want 
information on student behavior or attitude on the parent report. 

The survey found that 167 parents or 64% want both numeric and 
graphic summary of PALT test score information on the parent 
conference report. A third of the parents (N««86) want numeric test 
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score information; three percent of sample parents want a graphic 
summary of scores. Currently, test information is presented 
numerically; parents want scores and graphs to summarize student 
achievement. 

Over 90% of the responding parents (N=240) reported they would like 
a mailed Spring Parent-Conference Report. Currently, parents 
receive achievement information only at Fall Parent Conferences. 

Parents were asked what test score comparisons they want ; 49% of 
parents (N-134) said they want their child's test scores compared 
to the class average. Forty-four percent of parents (N=120) want 
their child's scored compared to the national average. Parents 
also expressed interest in comparisons to district (38%) , state 
(34%), and school (31%) averages. 

Approximately 70% of parents (N=182) use PALT test results to help 
them make decisions on their child's education or help them learn. 
Parents gave the following examples of decisions where test scores 
are used; helping students with homework/study skills; helping 
improve grades/ scores; and participation in TAG/special education. 

Sub-studv 4 - Grade 3 Testing Program Survey 

Subjects 

A non-random sample of 77 third grade teachers in 48 schools 
participated in a telephone survey on the grade 3 testing program. 

A telephone survey (Appendix D) was developed to ask grade 3 
teachers for input to improving the testing program. Teacher 
perceptions of fall/ spring grade 3 testing program were studied. 

Procedure 

A seunple of grade 3 teachers were contacted via telephone to 
participate in the survey. Telephone interviews were completed 
with 77 grade 3 teachers during a week in January 1987. 
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Results 

The results of the Grade 3 Testing Program survey Indicated that 
teachers wanted the Fall testing program to be similar to Spring 
testing. Teachers felt that students were able to handle the 
standard PALT answer sheet. The study set the direction for a 
grade 3 achievement testing task force and suggested revisions in 
the grade 3 testing program for Fall 1987. Appendix D presents a 
summary of the findings from the grade 3 telephone survey. 

DISCUSSION 

The link between evaluation use and effective educational practice 
has been well established (Griswold, Cotton, and Hansen, 1985) • 
Our belief is that if evaluative data are used effectively for 
decision making, there ought to be observable results in improved 
student performance. 

The first year of the decision making study helped us understand 
how PALT reports are used and the nature of the instructional 
decision process. It is hoped that the case studies planned for 
1987-88 will add to our knowledge, guide reporting system 
improvements, and improve the quality and use of evaluative data. 

The survey findings confirmed our hunches about the need and 
interest in training teachers and principals in evaluation use and 
revealed specific target areas for such training. A majority of 
principals (75%) and teachers (60%) are interested in evaluation 
inservice and technical assistance (TA) . Fifty percent of 
principals with 1-12 years experience are always/often interested 
in evaluation training; of principals with 13-25 years experience, 
60% are interested in evaluation TA sometimes. On the other hand, 
teachers, regardless of the years of experience, seem to have a 
predisposition either for or against data use. At all experience 
levels, 60% of teachers are interested in evaluation training and 
40% would seldom use evaluation TA. Levels of Use data also 
support this finding. Among the identified users of PALT reports, 
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32% are mechanical users of evaluation reports. This group needs 
training in the use and interpretation of test reports. 

Direct application of PALT student achievement information for 
instructional planning was indicated by 91% of the principals and 
58% of the teachers. Basically, principals believe that sharing 
data with teachers helps increase student achievement. After ten 
years of PALT reporting, the P-score is used more frequently than 
RIT scores by principals and teachers. This has implications for 
training. Teachers and test coordinators need additional training 
and support in the use of RITs. These results provide a basis for 
Departmental planning of direct services to schools. 

RECX}NMENDATIONS 

The findings of the first year of the Use of Evaluative Data for 
Instructional Planning and Decision Making study suggest the 
following recommendations: 

1. The RSE Department should revise the Parent Conference Report. 
Consideration should be given to including both numeric and 
graphic test score summaries. A revised Parent Guide to 
understanding test scores should accompany the report. 

2. The RSE Department should institute a Spring Parent Conference 
Report, including fall and spring PALT test scores. The report 
could be mailed to parents with the Parent Guide to testing. 

3. Based on early findings from teacher interviews and surveys, 
the RSE Department revised the grade 3 testing program. 
Prograua implementation and changes should be evaluated. 

4. The R&E Department should consider developing training 
materials/packages to help teachers prepare students for 
testing. Interviews identified this area as a priority 
interest ^f teachers and principals. 

5. The R&E Department should consider developing inservice 
training for test coordinators or others interested in test 
intezpretation and use of test results. 
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